1. Technical Field
The present disclosure relates generally to human-computer interfaces and mobile devices, and more particularly, to spatial motion-based user interactivity.
2. Related Art
Mobile devices fulfill a variety of roles, from voice communications and text-based communications such as Short Message Service (SMS) and e-mail, to calendaring, task lists, and contact management, as well as typical Internet based functions such as web browsing, social networking, online shopping, and online banking. With the integration of additional hardware components, mobile devices can also be used for photography or taking snapshots, navigation with mapping and Global Positioning System (GPS), cashless payments with NFC (Near Field Communications) point-of-sale terminals, and so forth. Such devices have seen widespread adoption in part due to the convenient accessibility of these functions and more from a single portable device that can always be within the user's reach.
Although mobile devices can take on different form factors with varying dimensions, there are several commonalities between devices that share this designation. These include a general purpose data processor that executes pre-programmed instructions, along with wireless communication modules by which data is transmitted and received. The processor further cooperates with multiple input/output devices, including combination touch input display screens, audio components such as speakers, microphones, and related integrated circuits, GPS modules, and physical buttons/input modalities. More recent devices also include accelerometers and compasses that can sense motion and direction. For portability purposes, all of these components are powered by an on-board battery. In order to accommodate the low power consumption requirements, ARM architecture processors have been favored for mobile devices. Several distance and speed-dependent communication protocols may be implemented, including longer range cellular network modalities such as GSM (Global System for Mobile communications), CDMA, and so forth, high speed local area networking modalities such as WiFi, and close range device-to-device data communication modalities such as Bluetooth.
Management of these hardware components is performed by a mobile operating system, also referenced in the art as a mobile platform. Currently, popular mobile platforms include Android from Google, Inc., iOS from Apple, Inc., and Windows Phone, from Microsoft, Inc. These three platforms account for over 98.6% share of the domestic U.S. market.
The mobile operating system provides several fundamental software modules and a common input/output interface that can be used by third party applications via application programming interfaces. This flexible development environment has led to an explosive growth in mobile software applications, also referred to in the art as “apps.” Third party apps are typically downloaded to the target device via a dedicated app distribution system specific to the platform, and there are a few simple restrictions to ensure a consistent user experience.
User interaction with the mobile device, including the invoking of the functionality of these applications and the presentation of the results therefrom, is, for the most part, restricted to the graphical touch user interface. That is, the extent of any user interaction is limited to what can be displayed on the screen, and the inputs that can be provided to the touch interface are similarly limited to what can be detected by the touch input panel. Touch interfaces in which users tap, slide, flick, pinch regions of the sensor panel overlaying the displayed graphical elements with one or more fingers, particularly when coupled with corresponding animated display reactions responsive to such actions, may be more intuitive than conventional keyboard and mouse input modalities associated with personal computer systems. Thus, minimal training and instruction is required for the user to operate these devices.
However, as noted previously, mobile devices must have a small footprint for portability reasons. Depending on the manufacturer's specific configuration, the screen may be three to five inches diagonally. One of the inherent usability limitations associated with mobile devices is the reduced screen size; despite improvements in resolution allowing for smaller objects to be rendered clearly, buttons and other functional elements of the interface nevertheless occupy a large area of the screen. Accordingly, notwithstanding the enhanced interactivity possible with multi-touch input gestures, the small display area remains a significant restriction of the mobile device user interface.
Expanding beyond the confines of the touch interface, some app developers have utilized the integrated accelerometer as an input means. Some applications such as games are suited for motion-based controls, and typically utilize roll, pitch, and yaw rotations applied to the mobile device as inputs that control a on-screen element. Along these lines, more recent remote controllers for video game console systems also have incorporated accelerometers such that motion imparted to the controller is translated to a corresponding virtual action displayed on-screen. Accelerometer data can also be utilized in other contexts, particularly those that are incorporated into wearable devices. However, in these applications, the data is typically analyzed over a wide time period and limited to making general assessments of the physical activity of a user.
Because motion is one of the most native forms of interaction between human beings and tangible objects, it would be desirable to utilize such inputs to the mobile device for controlling user interface elements thereof. Movement over extended distances can be sensed with GPS and other navigation-related modalities, while conventional accelerometer-based sensing can be used for sensing small movements of relatively short duration. However, there is a need in the art for spatial motion-based user interactivity that bridges the gap between these different modalities to provide a more immersive experience. It would also be desirable implement such interactivity on mobile devices on a universal basis for third party app developers to utilize.